Papers with consistency regularization
An Empirical Study of Consistency Regularization for End-to-End Speech-to-Text Translation (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for speech-to-text translation (ST) have achieved impressive supervised and zero-shot performance. |
| Approach: | They propose to use consistency regularization methods to boost end-to-end (E2E) speech-totext translation (ST) by regularizing the intra-modal consistency instead of the modality gap. |
| Outcome: | The proposed training strategies achieve state-of-the-art (SOTA) performance in most translation directions. |
CORD: Balancing COnsistency and Rank Distillation for Robust Retrieval-Augmented Generation (2025.naacl-short)
Copied to clipboard
| Challenge: | Existing methods to ground large language models fail to adequately attend to all contexts . position bias is hindered by retrieval-augmented generation, which requires constant attention . |
| Approach: | They propose to augment and distill training instances with their perturbed positions to encourage consistent predictions . they also propose to balance COnsistency and Rank Distillation by combining noise-controlled perturbations with augmentation and distillation. |
| Outcome: | The proposed method outperforms existing methods in diverse RAG benchmarks. |
RAM-EHR: Retrieval Augmentation Meets Clinical Predictions on Electronic Health Records (2024.acl-short)
Copied to clipboard
| Challenge: | Existing deep learning models for EHRs rely on knowledge from a single source and do not capture the semantic information for medical codes. |
| Approach: | They propose a Retrieval AugMentation pipeline to augment clinical prediction on EHRs . they use multiple knowledge sources to convert them into text and use consistency regularization to capture complementary information from patient visits and summarized knowledge. |
| Outcome: | Experiments on two EHR datasets show that RAM-EHR improves clinical prediction tasks. |
Enhancing Cross-lingual Natural Language Inference by Soft Prompting with Multilingual Verbalizer (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to cross-lingual natural language inference lack annotated parallel corpora. |
| Approach: | They propose a new prompt learning framework with the Multilingual Verbalizer for XNLI that uses a multilingual verbalizer to align the representations of original and augmented multilingual questions into a unified semantic space with consistency regularization. |
| Outcome: | The proposed framework outperforms existing methods under few-shot and full-shot cross-lingual transfer settings. |
MultiMatch: Multihead Consistency Regularization Matching for Semi-Supervised Text Classification (2025.emnlp-main)
Copied to clipboard
| Challenge: | **MultiMatch** is a semi-supervised learning (SSL) algorithm that combines co-training and consistency regularization with pseudo-labeling. |
| Approach: | They propose a semi-supervised learning algorithm that integrates co-training and consistency regularization with pseudo-labeling. |
| Outcome: | The proposed algorithm outperforms the second-best approach on 8 out of 10 setups from 5 natural language processing datasets and outperformed the second best by 3.26%. |
Consistency Regularization for Cross-Lingual Fine-Tuning (2021.acl-long)
Copied to clipboard
Bo Zheng, Li Dong, Shaohan Huang, Wenhui Wang, Zewen Chi, Saksham Singhal, Wanxiang Che, Ting Liu, Xia Song, Furu Wei
| Challenge: | Experimental results show that consistency regularization improves cross-lingual fine-tuning . pre-trained cross-linguistic models can transfer task-specific supervision from one language to the other . |
| Approach: | They propose to improve cross-lingual fine-tuning with consistency regularization . they use example consistency regularized to penalize prediction sensitivity to four types of data augmentations . |
| Outcome: | The proposed method improves cross-lingual fine-tuning across tasks . it can be generalized to other target languages without additional training . |
EICO: Improving Few-Shot Text Classification via Explicit and Implicit Consistency Regularization (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for few-shot text classification are limited by labeled data. |
| Approach: | They propose to use consistency regularization to improve few-shot text classification by generating pseudo-labels from weakly-augmented and strongly-augmented views. |
| Outcome: | The proposed method achieves competitive performance with 16 labeled examples with prompt and verbalizer. |
Advancing Test-Time Adaptation in Wild Acoustic Test Settings (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing wild vision TTA methods fail to handle speech data due to the unique characteristics of high-entropy speech frames, which are unreliably filtered out even when containing crucial semantic content. |
| Approach: | They propose a method for acoustic foundation models to perform confidence-based adaptation in wild acustic test settings. |
| Outcome: | The proposed method outperforms baselines under Gaussian noise, environmental sounds, accent variations, and sung speech in the wild. |
Simple Data Augmentation with the Mask Token Improves Domain Adaptation for Dialog Act Tagging (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on DA tagging focus on human-human social conversations, which is less applicable for task-oriented setting. |
| Approach: | They propose a controllable mechanism that augments text input by leveraging the pre-trained Mask token from BERT model. |
| Outcome: | The proposed mechanism augments text input by leveraging the pre-trained Mask token from BERT model. |
Prosody as Supervision: Bridging the Non-Verbal–Verbal for Multilingual Speech Emotion Recognition (2026.acl-long)
Copied to clipboard
| Challenge: | Existing paradigms for low-resource multilingual speech emotion recognition rely on labeled verbal speech and lack cross-lingual transfer. |
| Approach: | They propose a paralinguistic supervision paradigm for low-resource multilingual speech emotion recognition that leverages non-verbal vocalizations to exploit prosody-centric emotion cues. |
| Outcome: | The proposed framework outperforms Euclidean counter parts and strong SSL baselines in the language-based evaluation of low-resource multilingual speech emotion recognition (LRM-SER) |